The Differential Entropy of Mixtures: New Bounds and Applications

Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering physics, and obtaining bounds on the entropy of mixture is fundamental importance many these applications. This article provides sharp concavity deficit, which difference between differential weighted sum entropies constituent components. Toward establishing lower upper results that their own right obtained. In order obtain nontrivial bounds, properties skew-divergence developed notions “skew” $f$ -divergences introduced; reverse Pinsker inequality bound Jensen-Shannon divergence obtained along way. Complementary derived with special attention paid case corresponds independent summation continuous discrete random variable. Several applications delineated, including mutual information additive noise channels, thermodynamics computation, functional inequalities.